Automated scoring systems may be used to automatically score a variety of different item types without a need for human intervention or requiring only limited human intervention. These systems may be configured to automatically evaluate, for example, multiple choice responses, essay responses, speech sample responses, and other types of responses elicited by a test prompt. Automated scoring systems may be used to mitigate time, cost, and reliability issues that may be inherent in human scoring systems. Notwithstanding these issues in human scoring systems, it may be desirable in certain situations for automated scoring systems to produce scores that are consistent with scores assessed by human graders. Thus, in certain situations, operation of an automated scoring system may be modified to add a level of scoring flexibility consistent with human graders' scoring methods.